Goto

Collaborating Authors

 Lexington County


Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essays

Lee, Jinsook, Alvero, AJ, Joachims, Thorsten, Kizilcec, René

arXiv.org Artificial Intelligence

People are increasingly using technologies equipped with large language models (LLM) to write texts for formal communication, which raises two important questions at the intersection of technology and society: Who do LLMs write like (model alignment); and can LLMs be prompted to change who they write like (model steerability). We investigate these questions in the high-stakes context of undergraduate admissions at a selective university by comparing lexical and sentence variation between essays written by 30,000 applicants to two types of LLM-generated essays: one prompted with only the essay question used by the human applicants; and another with additional demographic information about each applicant. We consistently find that both types of LLM-generated essays are linguistically distinct from human-authored essays, regardless of the specific model and analytical approach. Further, prompting a specific sociodemographic identity is remarkably ineffective in aligning the model with the linguistic patterns observed in human writing from this identity group. This holds along the key dimensions of sex, race, first-generation status, and geographic location. The demographically prompted and unprompted synthetic texts were also more similar to each other than to the human text, meaning that prompting did not alleviate homogenization. These issues of model alignment and steerability in current LLMs raise concerns about the use of LLMs in high-stakes contexts.


Neural-Symbolic Models for Logical Queries on Knowledge Graphs

Zhu, Zhaocheng, Galkin, Mikhail, Zhang, Zuobai, Tang, Jian

arXiv.org Artificial Intelligence

Answering complex first-order logic (FOL) queries on knowledge graphs is a fundamental task for multi-hop reasoning. Traditional symbolic methods traverse a complete knowledge graph to extract the answers, which provides good interpretation for each step. Recent neural methods learn geometric embeddings for complex queries. These methods can generalize to incomplete knowledge graphs, but their reasoning process is hard to interpret. In this paper, we propose Graph Neural Network Query Executor (GNN-QE), a neural-symbolic model that enjoys the advantages of both worlds. GNN-QE decomposes a complex FOL query into relation projections and logical operations over fuzzy sets, which provides interpretability for intermediate variables. To reason about the missing links, GNN-QE adapts a graph neural network from knowledge graph completion to execute the relation projections, and models the logical operations with product fuzzy logic. Experiments on 3 datasets show that GNN-QE significantly improves over previous state-of-the-art models in answering FOL queries. Meanwhile, GNN-QE can predict the number of answers without explicit supervision, and provide visualizations for intermediate variables.


Grey Models for Short-Term Queue Length Predictions for Adaptive Traffic Signal Control

Comert, Gurcan, Khan, Zadid, Rahman, Mizanur, Chowdhury, Mashrur

arXiv.org Artificial Intelligence

Adaptive signal control system (ASCS) is the most advanced t raffic signal technology that regulates the signal phasing and timings considering the traffic patterns in real-time in order to reduce traffic congestion. Real-time prediction of traffic queue length can be used to adj ust the signal phasing and timings for different traffic movements at a signalized intersection with A SCS. The accuracy of the queue length prediction model varies based on the many factors, such as th e stochastic nature of the vehicle arrival rates at an intersection, time of the day, weather and driver characteristics. In addition, accurate queue length prediction for multilane, undersaturated and satur ated traffic scenarios at signalized intersections is challenging. Thus, the objective of this study is to devel op short-term queue length prediction models for signalized intersections that can be leveraged by adapt ive traffic signal control systems using four variations of Grey systems: (i) the first order single variab le Grey model (GM(1,1)); (ii) GM(1,1) with Fourier error corrections (EGM); (iii) the Grey Verhulst mo del (GVM), and (iv) GVM with Fourier error corrections (EGVM). The efficacy of the Grey models is th at they facilitate fast processing; as these models do not require a large amount of data; as would be needed in artificial intelligence models; and they are able to adapt to stochastic changes, unlike stat istical models. We have conducted a case study using queue length data from five intersections with ad aptive traffic signal control on a calibrated roadway network in Lexington, South Carolina. Grey models w ere compared with linear, nonlinear time series models, and long short-term memory (LSTM) neura l network. Based on our analyses, we found that EGVM reduces the prediction error over closest co mpeting models (i.e., LSTM and Additive Autoregressive (AAR) time series models) in predicting ave rage and maximum queue lengths by 40% and 42%, respectively, in terms of Root Mean Squared Error (R MSE), and 51% and 50%, respectively, in terms of Mean Absolute Error (MAE).